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ABSTRACT 

A United States Employment Service (USES) General 
Aptitude Test Battery (GATB) plan was formulated to evaluate and 
develop additional job-related assessment methods. Eighteen 
alternative predictors were reviewed-, from which biodata was selected 
for de-elopment. The literature indicates that biodata provides 
increased validity with little or no adverse impact. The focus will 
be on writing that will generalize across an occupational cluster and 
on the identification of relevant constructs. Items will inquire 
about background information and life experiences and will be 
screened for bias. Various scoring methods will be reviewed and 
evaluated as research progresses. The plan includes administration of 
the experimental biodata instrument along with the GATB. Future 
results will demonstrate whether biodata, with or without the GATB 
improves validity aad reduces adverse impact. Five tables summarize 
testing plans. (Author/SLD) 
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Abstract 

A USES GATE Research Plan was formulated to evaluate 
and develop additional job related assessment methods. 
Eighteen alternative predictors were reviewed, of 
which, biodata was selected for development. The 
literature indicates biodata provides increased 
validity with little or no adverse impact. The focus 
will be on item writing that will generalize across an 
occupational cluster and on the identification of 
relevant constructs. Items will inquire about 
background information and life experiences and will be 
screened for bias. Various scoring methods will be 
reviewed and evaluated as research progresses. The 
plan includes administration of the experimental 
biodata instrument along with the GATB. Future 
results will demonstrate whether biodata, with or 
without the GATB, improves validity and reduces adverse 
impact. 
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The U,S, Employment Service 
The Wagner-Peyser Act of 1933 established the U.S. 
Employment Service (USES) as a federal-state program 
that helps employers find workers and job seekers find 
work. 

Test Research by U.S. Employment Service 

The U.S. Department of Labor (USDOL) , through the 
USES, in cooperation with State Employment Security 
Agencies (SESAs) has conducted occupational test 
research since the 1930s. Presently, there are four 
Test Development Field Centers (TDFCs) with 
corresponding Test Research Units (TRUs) . These are 
listed in Table 1. 



Insert Table 1 about here. 



This test research program has produced a variety 
of assessment instruments (see Table 2) . 



Insert Table 2 about here. 



^The National Academy of Sciences report, Fairness in 
Employment Testing (1989) , provides a detailed history to the U.S. 
Employment Service and the General Aptitude Test Battery. 
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The General Aptitude Test Battery (GATE) is widely 
used for vocational counseling and selection for 
specific jobs. 
History of GATE 

Since its publication in 1947, the GATE has been 
researched in a continuing program of development and 
occupational validation. It is a speeded test that 
measures nine aptitudes with 12 subtests. 

Research has produced over 450 Specific Aptitude 
Test Batteries (SATBs) , consisting of combinations of 
two^ three, or four aptitudes for specific occupations. 
Beginning in 1972, emphasis was given to revalidating 
existing SATBs using samples which include subgroups of 
minorities large enough to permit separate statistical 
analysis. This revalidation effort took place because 
the SATBs might have adverse impact on minority groups. 
Validity Generalization 

In 1980, USDOL contracted Dr. John Hunter of 
Michigan State University to conduct validity 
generalizatioi' (VG) studies on the GATE database. As a 
result, the USES implemented an experimental test-based 
referral system called the VG-GATB Referral System 
which made referrals on a top-down percentile basis. 
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Given that minority groups generally do not perform as 
well as the majority group on the GATB, referral based 
oi top-down percentiles adversely affected the 
employment chances of minorities. 

To counteract this effect, decided to convert 
aptitude scores into within-group percentile ranks. 
The resulting percentile scores reflected an 
applicant's standing with reference to his or her own 
racial or ethnic group. 
Justice Department Challenge 

In November of 1986, The U.S. Department of 
Justice challenged within-group scoring as reverse 
discrimination. The difference between the Labor and 
Justice Departments' opinions was the legality of 
extending race-conscious preferential treatment as a 
means of combating discrimination. Neither department 
disputed that there is a powerful legacy of 
discrimination to overcome. 

The Justice Department's challenge to within- 
group scoring was decided by the Civil Rights Act of 
1991 which addresses the issue directly: "It shall be 
an unlawful employment practice. . .to adjust the scores 
of, use different cutoff scores for, or otherwise alter 
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the results of, employment related tests on the basis 
of race, color, religion, sex, or national origin." 

However, DOL thought that within-group scoring was 
an extension of a referral policy negotiated in 1972 
among the Department of Labor, the Equal Employment 
Opportunity Commission, and the Department of Justice 
to avoid discrimination against minorities and to 
protect equal employment opportunity. 

Labor and Justice agreed not to halt use of the 
VG-GATB Referral System until a thorough study of it 
could be conducted by a body of experts. Thus, the 
National Academy of Sciences (NAS) was commissioned to 
review the GATE. 

The resulting NAS report (1989) , Fairness in 
Employment Testing; Validity Generalization, Minority 
Issues > and the General Aptitude Test Battery , made 
various recommendations. Based on one of these 
recommendations, a proposed USES-GATB research plan was 
foirmulated and published in the Federal Register , vol. 
55, No. 142 (July 24, 1990) stating "A more promising 
approach [of reducing score differences] appears to be 
developing and evaluating additional job related 
assessment methods such as biographical information." 
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Biodata Research Project 

In 1990, The Northern Test Development Field 
Center (NTDFC) in Detroit, Michigan, along with staff 
from the New York and Massachusetts TRUs, began a 
comprehensive literature revir'j on alternative 
predictors to the GATE, This led to an examination of 
18 alternate predictors. 

After a review, biodata (the use of biographical 
information to predict future performance) was selected 
because it appears to provide increased validity with 
little or no adverse impact. Subsequently, the 
clerical cluster of occupations was identified for 
development of the biodata instrument. 

Development of a biodata instrument does not mean 
the abandonment of the GATE, To the contrary, the GATE 
will be administered along with the experimental 
biodata instrument. The final instrument may be a 
combination of biodata and all, or parts, of the GATE, 
depending on which combination yields the best 
prediction and the least adverse impact. 
General izabilitv 

Subsequent literature review during 1991 provided 
evidence for generalizability , For example, Rothstein 
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et al. (1990) said, "Biodata instruments can be 
constructed and validated in a way that will lead to 
validity general izability . " (p. 182). 

According to the literature, to achieve 
generalizability across locations and within 
occupational groupings, the following steps are needed: 
(a) identify bread measures and abilities that would 
apply across occupations (Sharf , 1991) , (b) validate 
and cross-validate large samples 

(Brown, 1978), (c) identify and define broad criterion 
and reference groups (Mumford & Owens, 1987) , (d) 
conduct thorough job analysis (Mumford & Owens, 1987) , 
(e) consider the age range and likely antecedent 
behavior and experiences of the target population 
(Mumford Sc Owens, 1987) , (f ) conduct cross- 
organizational keying (Rothstein et al,, 1990), (g) 
screen item content for the range of jobs to be covered 
(Gandy et al,, 1989), and (h) develop job-generic 
criteria (Gandy et al,, 1989). 

Based on the literature review, the NTDFC made the 
following recommendations for the development of a 
biodata instrument. 

To begin, attention should focus on one or more 
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large, relatively homogeneous grouping (s) of 
occupations, such as clerical or health care. These 
groupings should be those that: (a) the Job Service 
handles now or is expected to handle in the future 
and/or (b) are among the projected "demand" 
occupations. 

Next, an analysis of the major duties and 
abilities common to each grouping should be undertaken. 
Finally, separate forms and/or scoring keys may need to 
be developed for different occupational groups. 

Biodata Instrument Development Plan 

A biodata development plan was originally written 
in August, 1991. Since then, the plan has undergone 
three revisions. The latest plan (see Table 3), 
targets November, 1994 as the completion date. 



Insert Table 3 about here 



The completed biodata instrument will include at 
least two alternate forms and a report on development 
and analysis. There are nine steps in the plan. Some 
of the steps overlap and will be conducted 
concurrently. The dates on the plan are the expected 
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completion dates for each step 

Steps 1 and 2, Develo£>ment of Research Plan, and 
Interview and Report of Experts Contacted, are 
completed. In step 2, the biodata team contacted and 
interviewed 19 experts in biodata instrument 
development. In December, 1991, a report summarizing 
the results of all the interviews, including 
recommendations for an advisory panel, was submitted to 
the national test development office in Washington, DC. 
Step 3, Advisory Panel Selection is still pending. 

Step 4, Item Writing Training has been completed. 
Step 5, for the job analysis, a cluster of clerical 
occupations has been targeted. A nationwide effort of 
data collection will be implemented and completed by 
April, 1993. 

Step 6, Criteria Development, is still pending. 
Multiple criteria are expected to be 

utilized. A global supervisory rating scale, similar 
to the scale used in GATB validity studies, is expected 
to be used as one of the criteria. Other measures 
under consideration are tenure, absenteeism, and salary 
increases. Objectives for selecting criteria include 
the following cited by van Rijn (1992) : must be valid. 
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reliable, fair, unbiased, realistic, relevant, 
understandable, inexpensive, and acceptable to 
management. 

Steps 7 and 8 include the data collection for and 
analysis of the pilot and experimental instruments. 
Finally, in step 9, a report of the results and 
administration manuals will be completed by August, 
1994. 

Construct Development 
From the literature review and contacts with 
experts, the NTDFC identified 75 potential constructs 
that could be used for a biodata instrument. A 
definition of the term, construct, adapted from 
Klimoski (1991) follows: 

A construct [is] a concept that has been 
deliberately created or adopted for a scientific 
purpose. 

The term predictor construct [means] some 
aspect of a person which . . . has relevance to 
predicting future behavior or performance. While 
the phrase implies an attribute of a device or 
measure, it usually . . . [references] a factor 
[to distinguish] among individuals, (p. 4) 
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In addition, the literature suggests that the 
ideal number of constructs for an instrument should be 
between 10 and 20. Accordingly, the original list of 
75 constructs was rationally reviewed and reduced to 
16. A lengthy process of combining closely related 
constructs, rewriting and condensing definitions,, 
renaming certain constructs, deleting and adding others 
resulted in the list of 10 shown in Table 4. 



Insert Table 4 about here 



Due to the overlap of constructs identified in the 
June 28, 1991, report of the Secretary's Commission On 
Achieving Necessary Skills (SCANS) (USDOL, 1991) , 
attempts were made to include as many of the 
Competencies listed in that report. 

After rationally selecting the constructs, 
definitions were written. A standard fonaat was used 
for each definition, modifying the model used by 
Peterson et al. (1990). The model used has three 
sentences: The first defines the construct; the second 
describes the attributes of a person who scores high on 
the trait; and the third describes the attributes of a 



Biodata Instrument Development 

13 

person who scores low on the trait. 

Employer Contacts 
Employers will be selected for data collection 
based on (a) availability of occupation being 
researched, (b) size, and (c) location. A database of 
potential employers will be maintained. If selected 
employers meet the design requirements and agree to 
participate in the study, cooperation will be solicited 
for: (a) job analysis, (b) instrument administration, 
and (c) criterion collection. The goal is for the 
employer to participate in all three phases. 
Job Analysis 

Concurrently with construct development, solecting 
a job analysis method and developing a job analysis 
inst ^ument to verify the constructs was undertaken. 
Seven methods were reviewed. (McCormick, 1979) . 

A task inventory measuring relative frequency and 
importance of tasks was chosen. A preliminary job 
analysis instrument was drafted by combining clerical 
task statements from three existing sources and 
adapting the scoring formats. Task statements were 
alphabetized, as suggested in McCormick (1979) and 
nuinbered sequentially. Two response scales were 
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written: Time spent doing and Importance to job. The 
response options were 0 through 4 for each scale. 

A draft was administered to a sample of employed 
clericals. Subsequently, the instrument was revised to 
eliminate redundancies and improve readability. In 
addition, items were grouped according to task and 
behavior elements, as recommended by F.W. Erwin 
(personal communication, Apri"*. 6, 1992). The response 
options were reduced to 0 through 3. for each scale 
because options 1 and 2 were barely distinguishable 
from each other. This format closely resembles that of 
the Job Requirements Questionnaire for Office and 
Related Classifications published by Richardson, 
Bellows, and Henry Company (1985). 

A preliminary analysis to identify patterns of 
responses and verify construct selection has been 
undertaken, but not yet completed. 

Item Writing 

In January, 1992, six analysts from the NTDFC and 
six analysts from the other TDFCs and the national 
office attended a three day seminar in background data 
and item writing. The seminar was conducted by Dr. 
Michael Mumford and his staff at George Mason 
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University, in Fairfax, Virginia. 

Itera writing began following the seminar. A six 
member biodata team from the NTDFC has been meeting 
regularly since then to generate construct-related 
items. Each item describes life experiences that 
parallel the behavior or job tasks described within the 
construct definition. 

Items are written using one of the seven formats 
(Mumford, 1992) shown in Table 5. 



Insert Table 5 about here 



The items are reviewed for (a) conciseness, (b) 
bias, (c) readability or colloquial expression, (d) 
pleasant and neutral connotation, (e) relevance to the 
construct, and (f) fakability. Items are accepted, 
revised, rejected, or assigned to another construct. 
The order of all the responses or options is checked 
for consistency. This is important to avoid confusing 
examinees and to avoid possible scoring problems. 

Analysts who attended the item writing seminar 
from the three other TDFCs and the national office will 
assist in writing items. This process of writing. 
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screening, and rewriting will continue until over 1,000 
items have been generated. The large number of items 
is necessary to accommodate alternate forms. 

Some consideration will be given to reviewing 
items from existing biodata item pools. Items from 
other sources will be used when the relevant behavior 
of a construct is difficult to describe and/or 
lacks the 50-60 items per construct needed for the 
pilot instrument. Among other possible sources for 
item generation are: (a) Catalogue of Life History 
Items (Glennon, et al., 1966), (b) 2,000 Scaled Items 
(Uhrbrock, 1961) , and (c) Development and Use of the 
Weighted Application Blank (England^, 1971) . 

Over 700 items have been tten. After 1,000 
items have been written and revised, they will be 
submitted to two expert panels: a review panel, 
consisting of representatives from the Equal Employment 
Opportunity, Civil Rights and Legal Departments in the 
State of Michigan, and an advisory panel consisting of 
the experts experienced in biodata instrument 
development and use. 

Both panels will screen the content of each iteiU 
for the following five principles: (a) non-bias and 
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non-stereotyping, (b) job relevancy, (c) invasion of 
privacy, (d) events under the control of the job 
applicant, and (e; verif lability (Gandy, et al. 1989), 

By August, 1993, the items will be revised and 
pilot-tested nationwide. After data collection, item 
'Analysis will determine which items are non-predictive, 
biased, or inappropriate. 

The experimental instrument will be developed with 
a minimum of 30-40 items for each construct. The 
instrument will be administered to a national sample. 
After the final data analysis, the remaining items will 
be divided to create alternate forms of the biodata 
instrument. Each form will consist of 100-150 items. 

Scoring 

When data collection and item analysis are 
completed, a scoring method will be chosen. The 
instrument will be empirically keyed. This is the 
traditional approach to scoring biodata. 
There are a variety of scoring methods: 

1. Horizontal Per Cent: The simplest and most 
commonly used technique where the response is weighted 
proportionally to the probability of success. 

2. Vertical Per Cent: This methoa is also 
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commonly used and depends on the discriminating power 
of the response* High and low criterion groups are 
designated, and weights are assigned based on response 
differences. The weight for any category is a function 
of the significance of the difference between 
percentages computed for both groups. 

3. Reciprocal Averages: In this method, weights 
for an item are proportional to its correlation with 
the composite or to its loading on a common factor. 
This is done to satisfy two requirements: (a) The 
weight assigned to any response category is the average 
of the total scores of all people responding in that 
category and (b) each individual's total score is the 
average of the weights assigned to his/her responses. 

4. Keying by Patterns of Response: This method 
groups items by content. Within each group, each item 
is correlated with the criterion. Phi coefficients are 
computed for items not rejected. 

5. Differences in Percentages: This method is a 
modification of the vertical per cent method that is 
utilized with Strong's tables (England, 1971). High 
and low criterion groups are established and assigned 
net weights. 
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6* Rare Response: In this method weights are 
assigned to items based on their frequency of response 
in the total applicant sample. 

7. Correlation Coefficients: Biserial 
correlation is used for relating a continuous, normally 
distributed variable with a dichotomous variable having 
an underlying normal distributic i; Point Biserial 
correlation is used for relating a continuous, normally 
distributed variable with a genuine dichotomy. 

8. Option-keying strategy: Each item response 
alternative is analyzed separately and contributes to 
the score only if it correlates significantly with the 
criterion (Kluger et al., 1991). 

There are other methods of scoring not mentioned 
here and each scoring approach has advantages and 
disadvantages. According to Guion (1965) , depending on 
the purpose of the biodata instrument, one method of 
scoring may be more appropriate than another. This is 
further confirmed by Aamodt and Pierce, Jr. (1987) , who 
state that problems scoring items with the Rare 
Response method are too numerous to use for all jobs. 
They also mention that the method chosen to weight 
biodata items is important and will affect the outcome 
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of the validation procedures. 

Once a scoring method has been selected, 
statistical analysis is conducted. To analyze the 
data, the Statistical Package for the Social Sciences 
(SPSS) will probably be used. 

Item Bias 

Differential item functioning is referred to in 
most of the literature as item bias or item 
discrepancy. Berk (1982), states that item bias 
studies are necessary, but may not be sufficient for 
all score inferences and uses. He also mentions that 
bias of any kind is undesirable, and every effort 
should be expended to minimize bias. Two principal 
approaches have been used to study item bias: 
judgmental and statistical « The two approaches serve 
as types of quality control in the test development 
process to provide subjective and/or objective 
information to modify or eliminate biased items. The 
literature places enough emphasis on item bias analysis 
that a careful look at this process is warranted, 
particularly for biodata development. 

Validation 

Research shows that biodata has high validity for 
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predicting a wide range of criterion measures. 
According to van Rijn (1980) , biodata has proven to be 
one of the best predictors of future behavior. There 
are three validation models: (a) criterion-related 
validity, (b) content validity, and (c) construct 
validity. A combination of these measures is likely, 
but the optimum combination should be based on needs, 
purposes, and contexts for which the instrument is 
intended. 

Reliability 
Van Rijn (1992) mentions that test-retest 
reliability estimates are more suitable measures of 
biodata 's reliability than other estimates based on 
internal consistency. Mumford and Owens (1987) also 
note that the combination of low item intercorrelations 
and high retest reliability permits the use of a few 
biodata questions to capture a lot of information about 
an applicant. In any situation, the selection of test 
items to increase reliability will tend to increase 
validity (Henrysson, 1971) . 

Computer Adaptation 
A paper and pencil scannable form will be 
developed for testing groups, along with a computerized 
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version for self-administration. A self-administered 
version enables the applicant to quickly complete the 
questionnaire, reducing administration time, lowering 
costs, and increasing efficiency and flexibility. This 
will also simplify scoring and improve security of 
keys. The instrument will probably be contained on one 
disk(s) and/ or loaded on hard disk drive. 

Future Research 
Research will continue on the biodata instrument 
for other occupational clusters. Health care 
occupations is being considered for the next cluster. 
With the experience from the first endeavor in the 
clerical area, instrument development for succeeding 
clusters should go more quickly. 



Biodata Instrument Development 

23 



Table 1 

Test Development Network 
National Office: 

Washington, DC 
Northern Area: 

Michigan (Northern Test Development Field Center) 
Massachusetts Test Research Unit 
Illinois Test Research Unit 
New Jersey Test Research Unit 
New York Test Research Unit 
Pennsylvania Test Re^isarch Unit 
Southern Area: 

North Carolina (Southern Test Development Field 
Center) 

Alabama Test Research Unit 
Arkansas Test Research Unit 
South Carolina Test Research Unit 
Tennessee Test Research Unit 
Texas Test Research Unit 
Virginia Test Research Unit 
Washington, DC Test Research Unit 
Western Area: 

Utah (Western Test Development Field Center) 
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Table 1 (Continued) 
Test Development Network 

Arizona Test Research Unit 
Colorado Test Research Unit 
New Mexico Test Research Unit 
Oregon Test Research Unit 
California (California Test Development Field 
Center) 
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Table 2 

USES Assessment Instruments 
General Aptitute Test Battery (GATE) 
Nonreading Aptitude Test Battery (NATB) 
Bateria de Examenes de Aptitud (BEAG) 
GATE For The Deaf 

Basic Occupational Literacy Test (BOLT) 
Wide Range Scale 
Interest Checklist 
Interest Inventory 
Clerical Skills Proticiency Tests 
Typing test 

Statistical typing test 
Automated typing test 
Dictation test 
Spelling test 
Legal spelling test 
Medical spelling test 
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Table 3 

Biodata Instrument Development Plan 

Step 1 Research Plan 8/91 
Step 2 Interview and Report on Experts 

Contacted 12/91 

Step 3 Advisory Panel Selection Pending 

Step 4 Item Writing Training 1/92 

Step 5 Job Analysis - Clerical Occupations 10/92 

Step 6 Criteria Development 12/92 

Step 7 Pilot Instrument Form 11/92 

A. Data Collection 

B. Item Analysis 

Step 8 Experimental Instrument TBA 

A. Data Collection 

B. Data Analysis 

Step 9 Report and Manuals TBA 
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Table 4 

Remaining 10 Constructs After March. 1992, Revision 

Communication 

Follow (s) Directions 

Information Processing 

Interpersonal 

Investigating 

Motivation 

Reasoning 

Responsible 

Teamwork 

Versatility/Variety 



Table f 

Item Writing Formats 



From "Methodology Review: Principles, F^rocedure s , and Findings in 
the Application of Background Data Measures'' by M. D. Mumfoi^d and 
W. A. Owens, 1987, Appl ied Psychological Measu rement , 11 (1), p. 
1-31. Copyright 1987 by the Appied Psychological Measurement Inc. 
Reprinted by Permission. 



1. Y#i-No 

Hiv# you found your life to date to b% p\%mnnn\ and 
Mtisfying? 

2. Norvcontlnuum, «lngl# cholc# 

What wu your marital status at college graduation? 

a) Singta 

b) Marriad. no children 

c) Mirriad. one or more children 

d) Widowad 

a) Saparatad or dtvorced 

3. Continuum, singia cholc# 
What Is your waJght? 

a) iJnd^ 135 pounds 

b) 136 to 155 pounds 

c) 156 to 1?5 poorvJs 

d) 176 to 195 pourxis 
a) Ovar 195 pounds 

4. NonKKintinuum, muttipla choice 

Chack each of tt>« followlr>g from wtifch you have 
suffarad. 

a) AKargiaa 

b) Astf ama 

c) Ulcaft 

d) Epilapsy 

a) Haadachas 
f) Afthrifia 

fl) Gastrohitastinal upsets 

h) High blood prassura 

i) Loss of haaring 



5. Non-continuum, plus aacapa option 
Whan are you most likely to have a headacha'' 

a) When I strain my eyes 

b) Wtian \ don"! eat on schadule 

c) Whan I am under pres^ture 

d) January first 

a) Navar have headaches 



6. Contlnuurti, plus aacapa option 

What was your length of service In your most racant 
)ob? 

a) Lass than 6 rrxjnths 

b) Batw^n 6 months arxj 1 year 

c) 1 to 2 yaars 

d) 2 yaars or rr>ore 

a) No pravious full tlrrie }ob 



7. Common atam, multlpla continuum 

Ovar tf»a past 5 years, how rrxich hava you anfoyad 
tha following? (usa 1 to 4 t>ek)w) 

a) Loar»r>g or watchfr>g TV 

b) Raading 

c) Constnjctlva hobbies 

d) Homa ImprTvamant 
a) Outdoor racraation 

1) Vary much 

2) Soma 

3) Vary IKtfa 

4) Not at all 
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